Businesses around the world are increasingly recognizing the need to support continuous professional development and skills acquisition and foster lifelong learning in their workforce, in order to maintain competitiveness in the global marketplace. Many have recognized the potential benefits of online training and education: the flexibility offered by the web-based learning environment allows working adult learners to engage with course materials “any time, any place, any path, any pace”. Well-designed online training courses promise to make education and training available in ways that fit the busy work and life schedules of employees, and almost two decades of research into online learning has demonstrated that there is no measurable significant difference in learning outcomes between face to face and online learning modalities. In the corporate context, where exam results and course grades rarely exist as measures of learner achievement or effective online course design, it falls to training and education managers to identify reliable and valid approaches to evaluating both course design and learner performance, in order to demonstrate a significant return on the sizable investments needed to implement high quality online learning. Unfortunately, few easily implementable approaches exist. As has been noted, “in spite of the best efforts of organizations and the professional trainers' associations, there are significant problems in evaluating the true impact of [online] training”.
Similarly, while the value of feedback from and to learners in educational settings is well-established, most corporate training departments lack the tools to gather accurate learner feedback (direct or indirect) about their online learning experience or activities. Such information is critical in evaluating whether training courses are meeting corporate educational needs and goals. In the absence of evaluative tools that return meaningful and easily interpretable data, corporate training departments are more likely to deliver web based courseware that simply reflects budgetary restrictions, rather than appropriate and evidence-based instructional design.
U.S. Pat. No. 7,131,842 to Hollingsworth, et al. disclose a method for measuring the effectiveness of a school's educational program. The method involves observing classroom teaching of students at the institution, and analyzing the observed teaching to determine effectiveness based on teacher time on task, teacher instructional effectiveness, alignment of student work to standards, and breadth of coverage of student work. However, Hollingsworth, et al. is directed to analyzing teaching and student work a traditional classroom environment, and does not teach capturing user interactions with a course delivery interface during online learning. The method does not incorporate real-time and/or ongoing analysis, but rather, only analyzes collected data after the observation is complete.
U.S. Patent Application Publication No. 2011/0167013 to Pogue, et al. discloses a method of online classroom quality control involving monitoring the interaction between an online instructor and online students to determine quality of teaching. However, Pogue, et al. is primarily directed toward evaluating the performance of an instructor, and the criteria used for establishing performance include only pass rate, grades, survey results, student and teacher log-in amounts and participation. Pogue et al. is not directed to utilizing event capture to derive behavioral observations relating to course design, to granular response latency relating to dilemma and behaviors that serve an automatic function, or user interactivity with secondary resources.
U.S. Patent Application Publication No. 2010/0291528 to Huerta discloses a monitoring device configured to capture real-time participant events and stimuli in a learning environment. However, the data capture is primarily audio-visual, and the device is directed primarily to natural language interpretation of speech and analyzing biometric information in a live classroom environment to evaluate the performance of a teacher by identifying educational strategies. Huerta is not directed to an online learning environment, and does not teach capturing user interactions with a course delivery interface during online learning.
U.S. Patent Application Publication No. 2010/0190143 to Gal, et al. discloses a system which monitors, logs, and reports the performance of students based on their operation of molecular digital learning objects at a workstation. The system may determine whether objectives are completed within defined time periods, whether certain percentages of questions are completed correctly, or whether a student has not operated a keyboard or mouse for a particular time period. However, Gal, et al. is not directed to utilizing event capture to derive behavioral observations relating to course design, to granular response latency relating to dilemma and behaviors that serve an automatic function, or to user interactivity with secondary resources.
U.S. Patent Application Publication No. 2009/0035733 to Meitar, et al. discloses a computer program for monitoring the progress of tutors and students in a classroom environment. Meitar et al. teach allocation of learning activities to students stored data relating to scoring and timing of student testing. However Meitar et al. is not directed to utilizing event capture to derive behavioral observations relating to course design, to granular response latency relating to dilemma and behaviors that serve an automatic function, or to user interactivity with secondary resources.
A U.S. Patent Application Publication No. 2005/0053908 to Satheesh, et al. discloses a computer program for monitoring the progress of tutors and students in a classroom environment. Satheesh et al. teach the analysis of stored data relating to scoring and timing of student testing. However, Satheesh et al. is not directed to utilizing event capture to derive behavioral observations relating to course design, to granular response latency relating to dilemma and behaviors that serve an automatic function, or to user interactivity with secondary resources.